In panoramic stereo video systems, a multi-camera array with certain geometric configuration is often used to capture images. For example, the camera array may have 8 pairs of cameras installed on a mounting frame in the shape of a regular octagon, with each pair of cameras placed on each side of the octagon and having parallel optical axes. Such camera array can capture eight pairs of video data, with each pair capturing two videos to generate a stereo video. Two panoramic videos can be generated by stitching the videos captured by all the left cameras and all the right cameras in the eight pairs respectively. The two panoramic videos can then be sent to a display system to display a panoramic stereo video.
As compared with traditional single panorama stitching, stereo stitching faces more constraints. In particular, since the camera array often has a particular geometric configuration, the video data captured by the various cameras in the camera array will often have certain relationship among themselves that must be taken into consideration. Therefore, there is a need for a new method of stitching images to generate a stereo panoramic video that optimizes performance while satisfying certain constraints.